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Abstract 

Objectives:  State  agencies  and  Oregon’s  long-term  care  providers  cosponsored 
this  developmental  study  to  explore  the  creation  of  two  statewide  medication 
system  risk  models  using  sociotechnical  probabilistic  risk  assessment  (ST-PRA). 
This  paper  summarizes  the  methodology  involved  in  this  ongoing  project. 
Methods:  A  convenience  sample  of  18  facilities  participated.  Seven  multi¬ 
disciplinary  modeling  teams  used  process  mapping,  control  system  mapping, 
modified  failure  modes  and  effects  analysis,  and  ST-PRA  to  create  consolidated 
ST-PRA  models,  one  for  nursing  facilities  and  one  for  community-based  care 
(CBC) — i.e.,  residential  care/assisted  living — facilities.  Discussion:  The  models 
provide  contextual  maps  of  the  errors  and  behaviors  that  lead  to  medication 
delivery  system  failures,  including  unanticipated  risks  associated  with  regulatory 
practices  and  common  deviations  from  policies  and  procedures.  Policymakers, 
regulators,  and  managers  can  identify,  prioritize,  and  prospectively  model  risk 
reduction  interventions  using  ST-PRA.  Conclusion:  ST-PRA  models  can 
identify  systemic  and  behavioral  elements  that  increase  or  reduce  the  risk  of 
wrong-drug,  wrong-dose,  omitted-dose  or  -drug,  and  wrong-patient  medication 
administration  errors  in  nursing  and  CBC  facilities. 


Introduction 

Most  medication  errors  studies  have  been  done  in  the  acute  care  setting, 
whereas  comparatively  few  studies  have  been  done  in  long-term  care  (LTC) 
institutions.  ’  Information  about  the  medication  delivery  systems  in  LTC 
facilities  is  fragmented,  largely  drawn  from  retrospective  incident  analyses  or 
regulatory  surveys,  and  lacks  the  detail  necessary  to  design  systems  interventions 
to  reduce  the  risk  of  serious  errors  occurring  statewide.  The  Agency  for 
Healthcare  Research  and  Quality  (AHRQ),  through  its  Risk  Assessment 
Challenge  Grant  to  Oregon’s  public  health  agency  and  the  Oregon  Health  Care 
Association  (OHCA),  is  extending  the  application  of  modern,  prospective  risk¬ 
modeling  methods  to  the  identification,  prioritization,  and  mitigation  of  patient 
safety  risk  in  the  LTC  setting. 

OHCA  (Oregon’s  association  of  long-term  care  providers).  State  regulators, 
and  public  health  leaders  involved  in  the  design  of  this  study  are  seeking  to 
determine  whether  sociotechnical  probabilistic  risk  assessment  (ST-PRA)  can  be 
used  to  identify  common  systemic  and  behavioral  elements  that  increase  or  reduce 
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the  risk  of  serious  errors.  A  seeond  goal  is  to  determine  whether  these  models  ean 
be  used  to  design  statewide  risk-reduction  programs  for  nursing  and  community- 
based  care  (CBC)  long-term  care  facilities.  Four  classes  of  patient  events  (wrong- 
drug,  wrong-dose,  omitted-dose/drug,  and  wrong-patient  medication  errors)  were 
modeled  because  they  occur  frequently  and/or  have  potentially  serious  clinical 
consequences.'’"^  This  research  explores  the  feasibility  of  consolidating  input  from 
multiple  institutions  to  derive  two  Oregon  “master  risk  models,”  one  model  for 
nursing  facilities  and  another  for  CBC  (residential  care  facilities  with  more  than 
20  beds  and  assisted  living  facilities).  Residential  care  facilities  with  fewer  than 
20  beds  are  excluded  from  the  sample  because  they  are  often  quite  different  in 
scope,  structure,  and  operation  compared  to  the  facilities  in  this  sample. 

Until  recently,  health  care  did  not  have  the  tools  necessary  to  prospectively 
analyze  or  concurrently  compare  causal  mechanisms  for  different  types  of  adverse 
events.  This  situation  is  changing  as  health  care  begins  to  adopt  safety 
engineering  methods  from  the  nuclear  power  and  aviation  industries. 

g 

Introduced  by  accreditors  to  reduce  the  risk  of  future  errors,  root  cause  analysis 
and  failure  modes  and  effects  analysis  (FMEA)^  are  primarily  being  practiced  in 
the  hospital  environment. "  These  techniques  represent  the  most  basic  type  of 
causal  or  risk  analysis.  A  more  advanced  technique  that  models  multiple 
combinations  of  events  leading  to  a  single  outcome,  sociotechnical  probabilistic 
risk  assessment,  has  been  successfully  used  to  develop  prospective  risk  models  for 
hospital  medication  delivery  systems.  In  this  study,  ST-PRA  is  used  to 
consolidate  input  from  10  nursing  and  8  CBC  facilities  to  create  risk  models  that 
describe  potential  failure  combinations  resulting  from  processing  prescriber 
orders;  transcription  activities;  medication,  inventory,  and  storage  activities; 
pharmacy  dispensing;  and  medication  administration  practices  in  these  facilities. 
The  models  focus  on  processes  within  the  direct  control  of  the  facilities,  so  that 
potential  failure  combinations  resulting  from  pharmacy  internal  dispensing 
practices  are  not  modeled.  However,  pharmacy  dispensing  errors — internal  or 
external — that  enter  the  facility  are  captured  in  the  models. 

State  and  LTC  leaders  anticipate  that  these  risk  models — because  they  display 
and  quantify  the  complex  relationships  between  system  elements,  organizational 
culture,  behavioral  norms,  regulatory  requirements,  and  undesirable  outcomes — 
will  guide  development  of  realistic  medication  safety  policies,  identify  high-yield 
interventions,  and  substantially  accelerate  statewide  risk-reduction  efforts.  The 
models  will  be  validated  using  a  random,  stratified  sample  of  Oregon  nursing  and 
CBC  facilities  in  a  later  stage  of  this  research.  This  paper  presents  a  high-level 
description  of  the  model  building  phase  of  this  research.  The  final  report  will 
include  a  detailed  discussion  of  ST-PRA  methodological  issues  and  results  of  the 
validation  effort.  Readers  desiring  more  detailed  information  on  ST-PRA  are 
referred  to  Marx  and  Slonim.^ 
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Methods 

Design 

The  Oregon  Long-term  Care  Medieation  Risk  Modeling  projeet  is  a 
developmental  study;  it  is  the  first  health  eare  study  using  ST-PRA  to  ereate 
medieation  risk  models  for  multiple  LTC  faeilities. 

Sample 

A  eonvenience  sample  of  6  LTC  ehains  and  18  faeilities  partieipated  in  the 
study.  One  organization  operated  a  single  large  eampus  and  partieipated  in  both 
the  nursing  and  CBC  modeling.  There  were  10  nursing  faeilities  from  4  nursing 
ehains,  ranging  in  size  from  88  to  214  lieensed  beds.  Three  CBC  ehains  provided 
eight  faeilities:  five  assisted  living  and  three  residential  eare  facilities.  The  size  of 
the  assisted  living  facilities  (ALFs)  ranged  from  60  to  87  licensed  beds,  and  the 
three  residential  care  facilities  ranged  in  size  from  64  to  122  beds.  The  modeling 
teams  were  composed  of  trained  ST-PRA  facilitators,  1  or  2  research  staff,  and 
medication  staff  (nurses,  certified  medication  aides,  and  caregivers)  recruited 
from  the  1 8  facilities.  Over  the  course  of  3  months,  four  chain-specific  nursing 
facility  teams  and  three  CBC  teams  met  to  build  medication  risk  models  using  the 
ST-PRA  process. 

Overview  of  ST-PRA  Modeling 

The  central  tool  of  ST-PRA  is  the  risk  tree,  also  known  as  a  fault  tree.  An  ST- 
PRA  risk  tree  starts  with  a  top-level  event,  which  is  the  undesirable  outcome  (e.g., 
a  wrong  dose  delivered  to  a  patient).  Risk  modeling  teams  then  identify  the 
failures  that  link  together,  leading  to  the  top-level  event.  The  risk  tree  in  Figure  1 
illustrates  the  interrelationships  between  human  errors,  cultural  or  behavioral 
norms,  systems,  and  equipment  failures  that  together  may  lead  to  a  top-level 
event. 

This  risk  tree  illustrates  an  initiating  error  (order  taker  writes  down  the  wrong 
dose)  combining  with  a  read-back  of  the  order  that  fails  to  catch  the  error,  and 
three  other  capture  opportunities  that  do  not  catch  the  initiating  error.  In  the 
process  of  building  the  ST-PRA  models,  the  modeling  teams  used  Relex  software 
(Version  7.6;  Relex  Software  Corp.;  Greensburg,  PA)  to  identify  50-500  different 
combinations  of  failures  leading  to  a  particular  top-level  event  risk  being 
analyzed.  What  follows  is  a  description  of  the  major  steps  of  ST-PRA  as  applied 
in  the  Oregon  project. 

Major  steps  in  the  Oregon  risk  assessment  process 

1.  Organizing  risk  modeling  teams.  Facility  managers  nominated  staff 
based  on  each  person’s  knowledge  of  medication  systems  and  their 
ability  to  communicate  candidly  about  the  processes  and  behaviors 
that  could  lead  to  system  failures  within  their  facilities.  A  total  of  16 
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Figure  1.  Example  of  risk  or  fault  tree — dose  transcribing  error 


nurses  and  certified  medication  assistants  (CMAs)  participated  in  the 
nursing  facility  modeling  groups.  A  total  of  14  nurses  and  caregivers 
participated  in  the  CBC  modeling  groups.  Staff  members  from  the 
same  chain  were  grouped  together,  and  each  chain-specific  group  met 
four  times  for  approximately  3  hours.  Groups  typically  included  one  or 
two  staff  from  two  or  three  different  facilities.  Pharmacists  and 
physicians  associated  with  chains  participated  in  separate  focus 
groups,  reviewing  the  preliminary  models  and  providing  feedback 
about  their  accuracy  and  completeness. 

2.  Build  initial  process  maps.  Beginning  with  an  undesirable  outcome 
or  top-level  event,  each  modeling  team  developed  preliminary  process 
maps  that  represented  medication  delivery  processes  under  the  direct 
control  of  the  facility  (processing  of  prescriber  orders,  transcription, 
receipt/storage  of  delivered  drugs,  pharmacy  dispensing  errors, 
medication  administration)  for  each  of  the  four  classes  of  patient 
events  being  studied.  Process  maps  helped  establish  the  scope  of  the 
model  by  putting  bounds  on  the  processes  and  types  of  medications  to 
be  analyzed  by  the  teams. 

3.  Build  control  systems  maps.  Control  system  mapping  is  a  tool  for 
identifying  both  active  and  passive  controls  in  the  process  being 
modeled.  Active  controls  are  features  of  the  system  that  are  in  place  to 
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specifically  help  manage  the  risks  of  the  undesirable  outcome  under 
analysis.  For  example,  a  unit  dose  or  blister-pack  cards  are  active 
controls  that  reduce  the  chance  of  wrong-dose  errors.  Passive  controls 
are  those  features  of  the  system  that  help  control  the  specific  risk  under 
analysis,  but  that  exist  in  the  system  for  some  purpose  other  than 
controlling  the  risk  being  modeled  (e.g.,  the  shape  and  color  of  pills). 
The  purpose  of  identifying  these  controls  is  to  be  able  to  evaluate  them 
through  ST-PRA  modeling. 

4.  Identify  process  and  control  system  failure  modes.  In  the  Oregon 
model,  system  components  are  the  physician,  pharmacist,  nurse, 
records  clerk,  medication  aide,  and  resident.  While  we  rarely  refer  to 
humans  as  system  components,  they  are  components  in  the  ST-PRA 
risk  tree.  Once  the  process  and  control  system  maps  were  complete, 
the  modeling  teams  used  a  modified  FMEA  analysis  to  identify 
individual  failures  (errors,  at-risk  behaviors,  systems,  and  equipment 
failures)  that  could  lead  to  the  undesirable  outcome  under  analysis. 

The  FMEA  provided  the  source  data  for  building  the  initial  risk  trees. 


Table  1.  Examples  of  summary  failure  modes  and  effects  analyses 


Process 

Failure  mode 

Downstream  controls 

Failure  effect 

In-room  patient 
identification 

Nurse  enters 
wrong  room 

Visual  identification  of 
patient 

Arm  band  check 

Verbal  name  check 

Verbal  discussion  of 
medication  with  patient 

Medication  delivered  to 
wrong  patient  (note:  ST- 
PRA  will  model  failures  of 
downstream  controls  that 
might  capture  nurse 
entering  wrong  room 
before  medication  reaches 
the  wrong  patient) 

Physician 

ordering 

Medication 
order  written  in 
wrong  chart 

Nurse  review  of  order 

Pharmacy  review  of 
order 

Verbal  discussion  of 
medication  with  patient 

Medication  administered  to 
wrong  patient  (note:  ST- 
PRA  will  model  failures  of 
downstream  controls  that 
might  capture  the 
physician  error  before 
reaching  the  patient) 

5.  Build  initial  risk  trees.  With  failure  modes  in  hand,  the  risk  modeling 
teams  began  building  the  initial  risk  trees,  creating  “branches”  or 
“splits”  whenever  they  identified  a  significant  variation,  for  example, 
whether  medication  staff  pull  their  blister-packed  drug  cards  by  taking 
them  from  bins  presorted  by  administration  time,  or  use  the  medication 
administration  record  (MAR)  to  pull  individual  cards  sorted  by  patient 
name  or  room  number.  Risk  trees  went  through  multiple  iterations 
until  the  modeling  groups  were  satisfied  that  they  have  captured  the 
most  important  elements  of  the  process. 

6,  Collect  quantitative  denominator  data.  Denominator  data  for 
creating  rates  for  the  events  that  are  represented  in  the  risk  trees  were 
collected  from  participating  nursing  and  CBC  facilities.  The  research 
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team  analyzed  data  from  random,  stratified  samples  of  de-identified 
MARs  to  establish  nursing  and  CBC  per-order  and  per-dose 
denominators  for  error  rates.  Standardized  rates  for  the  average 
number  of  preseriber  orders  and  oral  doses  of  medieations  delivered 
per  staff  per  year  were  developed  from  these  MAR  analyses.  These 
rates  were  used  during  the  modeling  meetings;  for  example,  if  a  nurse 
said  she  saw  a  partieular  error  (sueh  as  an  omitted  dose)  twiee  a  week, 
the  team  used  the  denominator  data  to  turn  the  time  frequeney 
estimates  into  an  estimate  of  errors  per  dose.  Throughout  the  model 
building,  the  teams  eontinued  collecting  data  to  support  the  analysis. 

7.  Estimate  human  error,  at-risk  behavior,  control  systems,  and 
equipment  failure  rates.  Once  the  risk  models  were  well  developed, 
the  modeling  team  members  began  to  identify  and  estimate  data — 
based  upon  their  experience,  the  experience  of  others  where 
applicable,  published  rates,  and  human-reliability  expert  input — for 
rates  of  control  systems  failures,  at-risk  behaviors,  errors  per 
medication  order,  and  errors  per  oral  medication  dose.  During  the 
process-mapping  and  risk  tree-building  stages,  modeling  teams  found 
many  variations  in  process.  These  included  how  medication  bins  or 
drawers  were  organized  (by  shift,  by  time  of  administration,  or  in 
alphabetical  order  by  either  patient  names  or  drug  names),  how 
medications  were  delivered  (pop/pass/sign  versus  sign/pop/pass),  as 
well  as  the  use  of  medication  carts  versus  preparation  in  a  medication 
room.  The  modeling  teams  were  required  to  explore  the  relative  risks 
of  each  variation — some  being  assessed  directly,  other  variations 
requiring  risk  tree  modeling  to  determine  the  relative  merits  of  one 
variation  versus  another. 

The  teams  also  had  to  develop  rates  for  at-risk  behaviors,  such  as 
failure  to  check  the  MAR  when  administering  medications.  This  is  a 
commonplace  at-risk  behavior  not  easily  identified  in  postevent 
investigations,  particularly  in  terms  of  estimating  a  normative  rate  for 
a  particular  institution.  Nurses  and  caregivers  spend  long  shifts  getting 
to  know  the  patients,  their  diagnoses,  and  their  medications.  Despite 
policies  and  procedures  that  require  checking  the  MAR  during 
medication  administration,  medication  staff  admit  that  in  practice,  for 
a  variety  of  reasons,  they  fail  to  universally  accomplish  this  safety 
check.' 

The  teams  noted  that  even  if  pharmacies  or  medication  aides  entered 
the  time  on  blister-packed  oral  medications  to  operationally  fulfill  the 
“five  rights”  (right  patient,  right  drug,  right  dose,  right  time,  right 
route),  skipping  the  MAR  check  could  lead  to  a  wrong-drug,  wrong- 
dose  or  omitted-dose  error  if  the  MAR  contained  new  orders  to  add, 
change,  or  discontinue  drugs.  The  group  discussed  rates  of  failure  to 
check  the  MAR  based  on  different  ratios:  1  in  100  doses,  5  in  100 
doses,  or  50  in  100  doses.  The  models  were  calculated  based  on  the 
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team’s  rate  estimates.  The  focus  group  then  reviewed  the  probability 
estimates  and  either  agreed  with  what  the  model  identified  as  the  risks, 
or  went  through  the  estimation  process  again  until  they  reached  a 
consensus.  Through  this  repetitive  process,  the  interdisciplinary  team 
arrived  at  an  estimate  for  the  local  cultural  norm  for  checking  the 
MAR.  Experience  indicates  that  these  team  estimates  are  more 
accurate  than  rates  derived  from  event  data,  and  are  often  more 
accurate  than  rates  predicted  by  senior  management  within  the 
facility. 

8.  Run  the  quantitative  model  and  “cut  set”  reports.  After  the  rate 
estimates  were  put  into  the  model,  the  software  calculated 
intermediate-  (gate)  and  top-level  event  probabilities  (i.e.,  for  wrong 
drug,  dose,  resident,  or  omission).  PRA  software  will  generally 
produce  a  report  listing  the  probabilities  of  each  unique  combination  of 
failures,  a  “cut  set,”  that  could  produce  the  top-level  event.  The 
software  ranks  the  combinations  of  failures  according  to  their 
probabilities,  allowing  the  modeling  teams  to  focus  on  those 
combinations  of  errors  that  have  the  highest  probability  of  producing 
the  medication  error  being  modeled. 

9.  Compare  the  model  against  data  and  group  expectations.  Team 
members  continued  iterative  development  of  the  model  until  the  model 
represented  the  best  available  representation  of  the  risk.  Once 
completed,  the  chain-specific  models  were  shared  with  chain 
management,  pharmacy  representatives,  and  nursing  facility  medical 
directors  to  collect  their  observations  and  input. 

10.  Consolidating  risk  models.  This  project  was  designed  first  to  build 
chain-specific  models,  and  then  merge  the  models  into  consolidated 
models  of  nursing  and  CBC  facilities.  Very  early  in  the  project,  the 
research  team  found  that  medication  processes  and  rates  described  by 
each  team  were  similar  and  decided  instead  to  develop  only  one  model 
for  each  facility  type,  nursing  and  CBC.  Across  chains  and  facilities, 
the  critical  elements  of  taking  prescriber  orders,  transcribing  them, 
ordering  and  receiving  drugs  from  pharmacies,  storing,  and 
administering  medications  were  similar,  with  occasionally  important 
variations  requiring  new  branches  in  the  risk  trees. 

Modeling  medication  delivery  systems  risks  in  multiple  facilities 
mirrors  the  challenges  of  building  single-hospital  models,  where  input 
from  many  different  units  or  departments  must  be  accommodated 
during  the  model-building  process.  Also,  the  scale  and  politics  are 
different;  for  example,  direct  competitors  had  to  agree  to  share  their 
models  during  consolidation.  However,  the  processes  for  building 
these  models  are  essentially  the  same  as  would  be  followed  for 
individual  institutions.  Single  institutions  represent  far  fewer  logistical 
problems;  modeling  is  usually  completed  within  a  few  days  rather  than 
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weeks.  The  proeesses  followed  within  the  modeling  groups,  however, 
refleeted  the  typieal  ST-PRA  group  experience. 

Variations  identified  by  the  groups  required  judgment  calls  by  the 
modeling  team  (group  members  and  consultants)  to  assure  a 
reasonable  scope  for  the  project.  Many  low-risk  failure  paths  and 
many  inconsequential  variations  were  not  built  out  in  the  model,  but 
were  modeled  at  a  high  level.  Practice  differences  by  shift  or 
individuals  were  common  within  single  facilities,  as  well  as  procedural 
differences  between  facilities  within  the  same  chain.  Policies  and 
procedures  varied  by  chain,  for  example,  whether  a  chain  initially 
purchased  a  3-  to  14-day  supply  of  medications  to  bridge  the  gap 
created  by  delayed  preauthorizations  by  insurers.  Group  members 
believed  there  was  less  likelihood  of  “borrowing”  medications  to 
assure  continuous  therapy  if  this  policy  was  in  place.  This  variation  is 
included  in  the  models  because  it  could  raise  or  lower  the  cumulative 
risk  of  an  error  occurring.  If  an  event  was  identified  and  might  be 
worthy  of  future  study,  it  was  included  in  the  risk  trees  as  an 
undeveloped  event.  An  example  of  this  is  the  variety  exhibited  in 
borrowing  practices  when  drugs  are  not  available.  Hundreds  of 
variations  in  medication  delivery  practice  were  seen  across  the  sites, 
but  most  had  little  or  no  impact  on  serious  medication  errors  that  reach 
residents.  Variation  and  subsequent  judgment  calls  regarding  what 
should  be  modeled  are  common  in  aviation  and  nuclear  power  PRAs, 
as  well.'^’ 

Probably  the  most  difficult  task  associated  with  creating  these  models 
was  keeping  all  the  modeling  group  and  quantitative  data  organized 
and  accurately  recorded.  In  every  group  meeting,  at  least  one  and  often 
two  research  team  members  recorded  data.  With  seven  groups  running 
more  or  less  simultaneously,  the  research  team  had  to  devise  methods 
to  record  and  synthesize  large  amounts  of  information  over  relatively 
short  periods  of  time  by  using  spreadsheets,  the  risk  analysis  software, 
and  copious  note  taking. 


Discussion 

The  scope  of  this  paper  does  not  permit  exploration  of  the  underlying 
methodologies  brought  together  in  ST-PRA  modeling  techniques.  ST-PRA 
model  building  requires  knowledge  of  human  factors,  human  reliability, 
probability  theory,  group  dynamics^ — as  well  as,  in  this  particular  study,  a  dose  of 
public  policy.  In  the  best  of  circumstances,  probability  assessment  models  are 
subject  to  missed  failure  paths,  mischaracterized  dependencies  between  errors, 
and  misestimated  failure  rates.  Furthermore,  model  design  is  driven  by  the  need  to 
keep  the  models  to  a  manageable  size,  while  addressing  the  danger  that  process 
and  procedural  variations  that  may  lead  to  risk  could  be  left  out.'^’ 
Notwithstanding  these  inherent  limitations  of  the  probabilistic  risk  assessment 
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methodology,  we  believe  that  ST-PRA  offers  four  advantages  over  eurrent  risk 
management  methodologies. 

1 .  ST-PRA  provides  a  structure  and  process  for  gathering  sometimes 
highly  sensitive  information  about  policy,  procedure,  and/or 
behavioral  deviations  not  otherwise  available.  (See  the  discussion  in 
“Modeling  the  real  versus  the  ideal  medication  delivery  system.”) 

2.  The  models  provide  contextual  maps  of  the  errors  and  behaviors  that 
lead  to  system  failures  so  that  policymakers,  regulators,  and  managers 
can  identify,  prioritize,  and  prospectively  model  risk-reduction 
interventions  using  ST-PRA.  (See  the  discussion  in  “Understanding 
the  context  of  errors.”) 

3.  Models  are  dynamic;  they  are  designed  to  evolve  as  fresh  data  from 
new  studies,  patient  safety  reporting  systems,  or  facility  incident 
reporting  systems  are  used  to  refine  probability  estimates  for  different 
elements  in  the  models.^ 

4.  Policymakers  and  regulators  are  able  to  appreciate  the  unanticipated 
consequences  of  particular  enforcement  actions,  such  as  increased 
medication-borrowing  behavior  to  avoid  citations  for  “drug  not 
available”  or  the  time  pressures  introduced  by  interpretations  of  the 
Federal  “2 -hour  rule,”'^  governing  the  time  within  which  a  drug  is 
administered  to  a  resident.  (See  the  discussion  in  “Role  of  risk  models 
and  regulation.”) 

Modeling  the  real  versus  the  ideal 
medication  delivery  system 

Given  the  prospective  nature  of  ST-PRA,  modeling  teams  are  willing  to 
describe  and  discuss  common  deviations  from  policies  and  procedures  (P&Ps) 
that  they  would  otherwise  not  divulge,  for  fear  of  criticism  or  punishment.  For 
example,  rates  of  noncompliance  for  checking  the  MAR  before  giving 
medications  are  not  generally  reported  in  the  literature,  nor  are  there  any 
descriptions  of  the  behaviors  that  lead  up  to  borrowing  medications  from  one 
patient’s  supply  to  sustain  treatment  for  another  patient.  These  practices  are 
generally  forbidden  in  P&Ps,  but  this  study  found  that  they  occur  with  some 
frequency  to  cope  with  circumstances  that  are  often  outside  the  staffs  control. 
Given  that  the  ST-PRA  modeling  teams  were  making  assessments  of  future  risk, 
they  were  much  more  open  about  what  really  happens  in  the  administration  of 
medications. 

Modeling  teams  explained  that  most  borrowing  occurs  when  a  clinically 
important  drug  (such  as  an  antibiotic,  anticonvulsive,  anticoagulant, 
cardiovascular,  or  analgesic)  is  not  immediately  available  in  the  form  or  dose 
needed  by  a  particular  patient.  Concerned  about  the  impact  on  the  resident  of 
missing  one  or  more  doses  of  these  drugs,  a  staff  member  borrows  the  drugs  to 
assure  therapeutic  continuity.  Borrowing  also  occurs  when  insurer 
preauthorization  policies  delay  acquisition  of  clinically  important  drugs. 
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Sometimes  borrowing  oecurs  to  avoid  surveyor  citations  for  “drug  not  available” 
entries  in  the  MAR.  Borrowing  results  because  emergency  boxes  are  not  typically 
stocked  with  every  dose  and  form  of  a  drug;  preauthorization  approval  can  take  2 
weeks  or  more;  someone  can  forget  to  order  a  refill;  or  the  pharmacy  is  unable  to 
deliver  the  drug  on  time.  The  model  estimates  the  risks  associated  with  these 
types  of  practices.  Managers  and  policymakers  can  decide  if  the  size  of  the  risk 
requires  action  on  their  part  and,  if  so,  have  some  clues  about  where  to  intervene. 

Understanding  the  context  of  errors 

In  addition  to  providing  access  to  largely  unobtainable  behavioral  data,  risk 
modeling  provides  context  unavailable  in  single-event  investigations,  such  as 
root-cause  analysis.  Single-event  analyses  cannot  capture  the  range  of 
combinations  of  elements  that  might  operate  to  produce  the  same  outcome — they 
can  only  describe  what  occurred  in  the  event  under  study.  The  example  above, 
which  describes  borrowing  practices,  illustrates  that  there  are  multiple  reasons 
why  a  particular  behavior  or  event  occurs.  ST-PRA  offers  the  advantage  of 
modeling  several  variations  of  systems  and  their  human  components,  including 
behaviors  associated  with  specific  work  processes. 

During  this  study,  the  modeling  teams  were  able  to  visualize  the  “system”  of 
medication  delivery  through  defining  the  combinations  of  failures  that  lead  to 
delivery  of  incorrect  medications  to  a  resident.  They  learned  the  value  of 
redundant  control  systems  and  capture  opportunities  from  the  model.  For 
example,  repeated  checking  of  transcribed  orders  from  one  month’s  MAR  to  the 
next  substantially  reduced  the  probability  of  errors  involving  incorrectly 
transcribed  doses  or  drugs  reaching  the  resident.  Modeling  teams  and  managers 
can  use  the  models  to  see  the  interrelationships  between  component  reliability  and 
system  reliability — an  interrelationship  that,  if  ignored,  can  lead  to  blaming 
individuals  for  failures  outside  of  their  control.  Too  often,  when  adverse  events 
occur,  health  care  professionals  lack  the  time  and/or  resources  to  thoroughly 
understand  the  elements  and  risk  factors  that  led  up  to  the  incident,  resulting  in 
sometimes  unfocused,  punitive,  or  piecemeal  solutions.  Health  care  institutions 
tend  to  focus  only  on  the  breach — the  error  that  contributed  to  the  patient  event — 
and  ignore  the  important  contributions  of  the  system  and  component  failures  that 
preceded  the  individual’s  lapse.  This  is  both  an  unfair  and  unrealistic  approach 
to  error  management,  particularly  if  the  error  arose  from  systems  failures  outside 
of  the  control  of  the  individual. 

The  Oregon  risk  models  force  us  to  see  serious  medication  errors  in  the 
context  of  the  systems  that  produce  them,  leading  to  a  better  understanding  of 
both  systemic  and  human  component  safety  risks.  Having  these  models  provides 
Oregon  policy,  public  health,  and  LTC  leaders  with  a  common  mental  model  to 
establish  priorities  for  planning  and  evaluating  alternative  statewide  medication 
risk-reduction  strategies.  At  the  facility  level,  managers  can  use  the  rankings  of 
relative  risks  for  the  modeled  errors  to  determine  where  to  concentrate  limited 
resources  for  maximum  risk  reduction. 
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Role  of  risk  models  and  regulation 

Troyan  Brennan  has  argued  that  regulation  eannot  improve  quality  if  it 
depends  primarily  on  licensing  and  disciplinary  actions  to  achieve  what  he  calls 
“culling,”  or  removing  defective  performers.  Brennan  offers  five 
recommendations  for  improving  quality  through  regulation.  ST-PRA  risk  models 
can  be  used  to  achieve  or  inform  each  of  Brennan’s  recommendations: 

•  Help  regulators  integrate  new  knowledge  into  public  policy 

•  Support  the  development  of  shared  regulatory/provider  aims 

•  Encourage  provider  self-improvement 

•  Support  innovation  to  improve  quality 

•  Improve  outcomes  (risk)  measurements 

Unfortunately,  space  does  not  permit  an  in-depth  discussion  of  each  of  these 
recommendations;  however,  the  following  example  helps  to  illustrate  the  unique 
contributions  of  ST-PRA  risk  models. 

Oregon’s  public  health  and  regulatory  leaders,  in  partnership  with  the  ETC 
provider  community,  intend  to  use  the  models  to  develop  shared  goals  or  aims  for 
quality  and  risk  reduction  in  the  State.  These  models  provide  State  regulators  and 
policymakers  with  a  view  of  risk  that  can  inform  both  future  legislation  and 
current  enforcement  practices.  Eor  example,  regulators  were  unaware  that  the  so 
called  “2-hour  rule”'^ — currently  interpreted  by  Oregon  surveyors  as  meaning 
that  all  medications  (including  vitamins  and  nutritional  supplements)  must  be 
administered  in  the  “window”  of  1  hour  before  or  after  the  time  specified  in  the 
prescriber’s  orders — would  play  a  prominent  role  in  the  risk  modeling.  This  rule 
encourages  undesirable,  at-risk  behavioral  norms  or  short  cuts,  as  medication  staff 
delivering  large  volumes  of  medications  at  certain  times  of  the  day  become 
pressed  for  time.  Regulators  and  providers  now  have  an  opportunity  to  discuss 
these  findings  objectively  and  to  evaluate  how  to  reach  their  shared  goal  of  timely 
medication  administration  to  achieve  therapeutic  efficacy.  Through  the  model, 
Oregon  regulators  and  providers  can  begin  a  dialogue  to  analyze  the  alternative, 
associated  risks  using  an  evidence-based  approach  to  narrow  the  scope  of  the  rule 
by  asking:  Which  risks  are  increased?  Which  are  decreased? 

ST-PRA  as  a  tool  for  State-level  change 

Even  though  it  has  been  used  for  decades  to  make  very  important  policy  and 
operational  decisions  in  other  high-risk  industries,  probabilistic  risk  modeling  is  a 
new  tool  in  health  care. As  noted  earlier,  while  the  ST-PRA  methodology  has 
limitations,  it  is  uniquely  suited  to  prospective  risk  assessment  of  combinations  of 
events  leading  up  to  the  top-level  event  or  undesirable  outcome.  A  skilled  and 
knowledgeable  leader  or  team  is  required  to  guide  a  modeling  group  through  the 
ST-PRA  process,  gaining  group  member  trust  and  encouraging  the  candor 
necessary  to  discuss  deviations  from  policy  that  are  often  ignored  or 
underestimated  by  outsiders.  Model-building  teams  require  enough  time  to  carry 
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out  the  development  of  aceurate  and  eomplete  risk  trees,  which  are  critical  to  the 
success  of  the  ST-PRA.  In  this  particular  study,  the  underlying  common 
medication  processes  (ordering,  transcribing,  receipt/storage,  and  administration) 
were  similar  across  facilities,  and,  while  important  variations  were  identified,  they 
did  not  prevent  consolidation  of  each  chain’s  models  into  one  “master  model”  for 
nursing  facilities  and  one  for  CBC  facilities.  How  interfacility  and  interchain 
variances  are  addressed  during  model  building  requires  good  process  and  control- 
system  modeling,  judgment,  and  PRA  skills  on  the  part  of  the  team  leaders. 
Consolidated  models  would  have  been  large  and  relatively  unmanageable  if  each 
facility  had  unique  processes  in  place. 

This  type  of  State-level  assessment  requires  trust  between  all  parties,  a 
significant  commitment  of  resources,  protection  of  this  very  sensitive  data  to 
maintain  confidentiality,  consideration  of  potential  policy  or  legal  consequences 
to  participants,  and  an  agreement  that  the  model(s)  will  be  used  to  improve  quality 
rather  than  to  punish  the  provider  community.  The  public-private  partnering 
demonstrated  in  this  study  requires  trust  and  cooperation  between  the  State 
agency  and  provider  leadership. 

Finally,  through  this  research,  the  OHCA  and  State  of  Oregon  partnership  will 
have  a  representative  model  of  the  active  controls  LTC  organizations  can  put  into 
place  to  manage  the  risks  of  medication  error.  The  control-system  model  that 
accompanies  the  ST-PRA  will  provide  an  overall  mapping  of  the  strategies  in 
place  to  manage  medication  risk.  A  new  piece  of  these  controls  is  the  ST-PRA 
itself,  which  provides  guidance  to  system  managers  on  the  risks  inherent  in 
medication  delivery  systems  and  the  relative  contribution  of  each  active  control  to 
reducing  the  risk  of  medication  error.  Each  medication  adverse  event  investigated 
by  State  agencies  or  participating  facility  will  become  a  learning  opportunity  to 
inform  the  risk  model,  by  either  validating  the  model  or  adding  risk  elements  not 
discovered  in  the  initial  ST-PRA  development.  The  result  of  this  research  is  a 
“living  risk  management  system”  that  can  and  should  be  used  by  system  managers 
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to  manage  the  risks  of  medication  error. 


Conclusions 

This  paper  provides  an  overview  of  ST-PRA  modeling  and  how  it  has  been 
used  in  Oregon  to  create  multifacility  probabilistic  risk  models  describing  the 
systems,  controls,  and  behavioral  elements  that  can  increase  or  decrease  the  risks 
of  serious  medication  errors  or  omitted  doses  in  nursing  and  CBC  facilities.  These 
findings  are  important  because  most  errors  are  invisible  to  providers  and  patients, 
leaving  them  with  the  impression  that  errors  are  either  trivial  or  infrequent.^^  The 
modeling  process  reveals  important  systems  relationships,  unintended 
consequences  of  regulatory  enforcement,  and  important  risk-reduction 
innovations  that  merit  dissemination.  These  models  can  guide  LTC  leaders  and 
policymakers  toward  common  goals  and  can  accelerate  risk-reduction  efforts. 

ST-PRA  models  establish  the  behavioral  and  systems  context  that  either 
increases  or  decreases  the  probability  of  errors  occurring.  This  research  tool 
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estimates,  often  for  the  first  time,  the  frequency  of  important  policy,  procedural, 
or  behavioral  deviations  that  are  usually  missed  in  discussing  the  source  of 
medication  errors — for  example,  the  practice  of  borrowing  drugs  from  one 
resident’s  supply  to  give  to  another  resident  whose  drugs  are  not  yet  available. 
The  models  provide  important  insights  and  describe  causal  relationships  missing 
from  global  measures  sometimes  used  in  health  care,  e.g.,  studies  associating 
higher  or  lower  medication  error  rates  to  nurse  staffing  ratios.^"^  The  ST-PRA 
models  and  the  interventions  based  on  them  have  the  potential  to  become  the 
foundation  of  an  active  public-private,  statewide  medication  risk  management 

25 

program  focused  on  sustained  risk  reduction  and  learning  from  errors. 

If  the  Oregon  models  are  successfully  validated,  the  State  will  begin  to  move 
from  the  current  incremental,  institution-by-institution  change  mode  to  one  that 
employs  comprehensive  risk  models  to  guide  coordinated  private  and  public 
sector  practice  improvement  efforts,  substantially  accelerating  risk  reduction 
across  a  broad  segment  of  the  provider  community.  Through  the  risk  modeling 
process.  State  and  provider  community  leaders  can  see  which  system  attributes 
and  which  component  failures  (errors,  behaviors,  or  equipment  failures)  have  the 
most  impact  on  common  medication  errors  and  focus  their  resources  on 
improvements  likely  to  bring  the  most  benefit  to  Oregon’s  citizens. 
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